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ABSTRACT " . 

The Southern Regional Education Board (SREB) Task 
Force on Higher Education and the. Schools calls for states "to 
consider the feasibility of developing a regional. assessment of 
teacher selection techniques" (SREB , 1981). This recomme'ndat ion 
^reflects concerns about the «ee|d for: Q) more states to develop new 
~, t.eacher certification tests, and (2) applicants to be given a state's 
required test if they have already taken and passed another test 
required elsewhere. The Task Fdrce strongly endorses a policy of 
requiring minimal competency of *■ 1 — * — • - • 



teacher applicants on' content areas. 
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The issue of whether there is a relationship between content mastery 
and the ability to "put it over" continues to be debated. This 
examines the relationship between the Georgia Teacher Certification 
Tests (TCT) and the corresponding area tests on the National Teacher 
Examinations (NTE). A second part of the study examines the 
relationships between the TCT scores and ItFe rat^flfe on the Georgia 
Teacher Performance Assessment Instrument (TPAI), afrid the scores on 
the NTE and the TPAI ratings. The existing research (jiase is not 
consistent as to the relationship between teacber te^fctng and student 
criterion, or teacher performance assessment' and student outcomes.. 
Previous research and this study tend to support the idea that a 
teacher's knowledge base, as measured by a paper-and'-pencil test, 
does not have a linear relationship with the ability to "put it 
across." (Author/PN) 
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The improved selection of public school teachers has become an important 
policy objective throughout the Southern region. Indeed, the Southern 1 
states have been leaders in developing selection techniques to insure minimum 
competency of beginning teachers. Three states, Georgia among them, have 
developed their own tests for teacher applicants. The tests are developed 
from objectives related to the curriculum taught in the states' schools, 
with input by teachers within these states on defining the objectives such 
tests are to cover. ■ 

Several Southern states have required prospective teachers to take the 
National Teacher Examinations (NTE) f which were developed by the Educational 
Testing Service (ETS) (ETS, 1981). These examinations are norm-referenced, 
so that scores ^ individual students may be compared against the distribution 
of scores of all the students who have taken the test. Cut-off f scores 
have not yet been established by several Southern states using these NTE 
tests* 

PURPOSE ' 
Georgia is unique among all states since teacher education majors 
from public institutions have been taking both the National Teacher Examina- 
tions, which provide achievement measures in three areas, of teacher preparation 
(general education, professional education, and specialty areas) and the 
state-developed Teacher Certification Test (TCT), which assesses teaching 
field knowledge (Georgia Department of Education, 1981). Both test "content" 
areas. Additionally, Georgia's beginning teachers are assessed on their 
on-the-job performance with the Teacher Performance Assessment Instrument 
(TPAI) which assesses 14 competencies of teaohers. This data base presents 
a unique opportunity to analyze for one group of teachers the relationship, 
if any, between content-testing" results on a state-developed, criterion- 



referenced test and a nationally normed test, as well as the results of 
either test against the performance assessment of teachers on the Job; 
especially since the Board of Regents has recently dropped its requirement 
for students to take, the NTE. 

The SREB Task Force on Higher Education and the Schools in its report, 
The Need for Quality , calls for states "to consider the feasibility of 
developing a regional assessment of teacher selection .techniques" (SREB, 
1981). This recommendation reflects concerns about the need for (1) more 
states to develop new teacher certification tests, and (2) applicants to 
be given a state's required treat if they have already taken and paaaed 
another teat required elaewhere. The recommendation also mirrora concern 
on whe.ther the content to be maatered by teachera in any one atate dif^fera, 
^or ahould differ, from what ia needed in other atatea. 

The Taak 'Force atrongly endoraea a policy of requiring minimal competency 
of teacher applicanta on content areaa, deapite the continuing charge from 
aome quartera that content maatery haa little to do with the ability of 
a teacher to perform in the claaaroom. Thua, the iaaue of whether there 
ia a relationahip between content maatery and the ability to "put it over" 
continuea to be debated, and beara reaearching where data permit ita evalua- 
tion. Georgia, being the firat atate with a comprehenaive aaaeaament of 
beginning teacher performance akilla aa well aa of content maatery, providea 
the opportunity to analyze thia important queation. 

Research Questions 

The following queationa will be atudied in each of the content areaa 
. for which matching data exist: 

What is the relationship between the Georgia Teacher Certification 
Test raw score and the scaled score on the National Teacher Examination 
Area Test for a selected population? 



For the population subgroup which passes each Teaeher Certification 
Test, what are the corresponding scores on the NTE? 

What relationships exist between the scores on the NTE area test, 
NTE Common, NTE Professional Education, TCT, and the on-the-job 
performance rating (Teacher Performance Assessaent Instrument) 
for a selected population? , } 

V 'Y 

Related Literature 

Teacher Testing and Student Achieve— nt. Many of the studies on the 
relationship between teacher testing and student performance are characterized 
by methodological problems. Lins (1946) reported moderate correlations 
between student achievement and NTE Common scores of teachers, but small 
samples were used. Significant relationships between NTE Common scores 
of teaohers and the achievement of their students in vooabulary and mathematics 
were reported by Sheehan and Marcus (1978), although they cite a study , 
showing an inverse relationship between NTE and pupil achievement. 

Teacher Testing and Performance Evaluation. Several studies have 
examined the relationship of the National Teacher Examinations to on-the- 

Job performance. Most of, the research studies have used ratings by supervisors 

c 

or principals of student teaohers or classroom teachers. Results have 

'i' 

bean mixed, but generally low correlations have been observed (Quirk, Hitten, 
and Weinberg, 1973)* One study, which looked at weighted common scores, 
selected area examinations, and supervising teachers' ratings, reported 
bath positive and negative correlations that were significant, but very 
few in number. The authors questioned the rating instruments that were 
used (Andrews, Blackmon, Davidson and Maokey, 1982). Medley and Hill (1970) 
studied the relationship between the Common subtest soores of the NTE and 
teaohing style, using a low inferenoe instrument, and found correlations 
with a median of .25- (Low inferenoe instruments measure the presence 
or absenoe of behaviors of the teaoher in the classroom; high inferenoe 



instruments use some type of rating system of these behaviors based on 

judgments of the raters,) Piper and Sullivan ( 1 98 1 ) report a significant ^ 

correlation of .43 between soores on the - blB elementary, education test 

with ratings of university supervisors during studept teaching. The results 

of the research are mixed, but generally support the view of ETS (1978) 

1 

that the NTE is not a predictor of classroom performance. 

Performance Assessment and Student Criterion. Coker, Medley and Soar 

1980) report mixed results in a study of student achievement and attitudes 

related to low inference teaching measures. Their data indicate that behav- 

iors "thought" to characterize the effective teacher may not be true. 

This idea is supported in the work of Medley (1977) and Rosenshine (197&). 

Capie (1981) , reports results of various studies examining the relationship ^ 

of the Georgia Teacher Performance Assessment Instrument to student achievement. 

He reports significant correlations between the TPAI ratings and achievement 

gains as measured by teacher-made tests. Mixed results are reported with 

the TPAI when student achievement is measured by standardized tests. 

Performance Assessment Instrumentation. What is the relationship 

between various types of instruments used to assess teacher performance? 

Ratings of teaoher performance historically have been suspect, and results 

of correlations of various systems with different criteria have yielded 

no oonolusions (Quirk et al. , 1973). With the assumption that teachers 
' i. *~ 

do affeot pupil learning, and that the learning is somehow related to what 

the teaoher does in the classroom, different types of assessment systems 

have been devised. 

The TPAI is the first statewide effort at employing a performance 

assessment with data polleotors carefully trained to make observations. 

Interrater agreements are Reported to be in excess' of .80 for in-the-classroom 
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observations (Capie, 1981). In a comparison of the /TPAI with an instrument 
developed by Coker (interrater agreements reported to be •80)(Coker and- 
Coker, 1982) , researchers observed 18 competencies of student teachers 
and found the correlation on the two instruments to be virtually zero, % 
leading the researchers to conclude that teacher observation is highly*" 

dependent on the instrument (Dickson and Wierama, 1980). 

-> 

METHODOLOGY * 

Design 

This is an ex post facto study (Kerlinger, 1973) examining the relationship 
between the Georgia Teacher Certification Tests (TCT) and the corresponding 
area tests on the National Teacher Examinations (NTE). A second part of 
the study examined the relationships between (1) the TCT scores and the 
ratings on the Georgia Teacher Performance Assessment Instrument (TPAI), 
and (2) the scores on the NTE and the TPAI ratings. 
The variables of the study were: 

1. The raw scores on the Georgia Teacher Certification Test. 

2. The scaled scores on the National Teacher Examinations area tests, 
the weighted Common Examinations score, and scores on the subtests 
of the Common Examinations. 

3* The Teacher Performance Assessment Instrument results expressed 
as the percentage of indicators above or at the minimum level for 
each of the 14 competencies. 

Populations 

In some cases students took tests more than once. Only scores for 
the first admnistration were used. Data were analyzed separately for each 
of the subjeot areas. Students who took tests in other areas were not 
included because no comparable tests existed for a particular area or numbers 
were less than 30 students. 
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Population for the study of the relationship of the area aoorea on 

the TCT to the area test scores on the NTE. The total population (ns223D 

consisted of graduates of the University System of Georgia between the 

years 1978-1980, who had taken the National Teacher Examinations from 1978- 

1980, and who also tfad taken the Georgia Teaoher Certification Test during 

the years 1978-1981 in one of the following areas: art (ns107), business 

(n=121), communicative arts (n=98), early childhood (n=828), home economics 

(n=9D> health and physical education (n=327), industrial arta (n=32), 

mathematics (n=6D, mental retardation (n=27M) , music (n=128), and social 

x 

studies x (n=l6M). Corresponding. tests on the NTE were: art education, 
Business education, English language and literature, early childhood education, 

hotiMSC; economics education, physical education, industrial arts education, 

% 

mathematics education, education of the mentally retarded, music education, 

•V* . 

and social studies. > 

Population for the study of the relationship of the TCT to the Teacher 
Performance Assessment results. The population (n= 1 1 15) for this part 
of the research consisted of thoee persons who were first year teachers 
in a public school in Georgia during the 1980-81 school year, who were 
required to be assessed for certification, and who had taken the TCT during 
tYte years 1978-1981. Areas included were: art (n=35); communicative^ acts 
(n=8W); early childhood (n=**52); home economics (n=35); health and physical 
education (n=138); mathematics (n=42); mental retardation. (n= 122f) ; music 
(n=62); social studies (n=76); and science (^=68). 

Population for the study of the relationship of the NTE tests to the 
Teacher Performance Asaesnoent results . The population (n=305) for this 
part of the research consisted of those persoiw* tf *who were first year teachers 
in a public school during the 1980-1981 school year, who were required 



to be assessed for certification, who had graduated from th* University 
System of Georgia 1978-1980, and who had taken the NTE examinations. Areas 
included were: early childhood education (n=179), education of the mentally 
retarded (n=6D, and physical education (n=6*5). 

Data Collection * 

All data were treated with utmost security. No scores or individual 
results were reported at any time. The data base for the project included: * 

1. Computer data base of the University System of Georgia, which 
contained NTE scores. 

2. Computer data base of the Georgia State Department of Education, 
which contained TCT scores and the results of the Teacher 
Performance Assessment for the population. 



Instrumentation 



National Teacher Examinations . The tests ar^ composed of 25 Area 
Examinations and the Common Examinations. The Cornet Examinations consist 
of a test in professional education, and one in general education (the - 
latter has 3 subparts): written English expression; social studies, literature, 
and the fine arts; and science and mathematics. A Weighted Common Examinations 
Total (WCET) is a combination of the above tests, with professional eduaation 
receiving a weighting of *1.0; social studies, 2.5; written English expression, 
1.0; and science and mathematics, 2.5. The sum of the products is the 
total weighted score (WCET) with possible values ranging from 250- A. 

The Area Examinations test the content of an undergraduate special 

field oh major in undergraduate education. Area test scores range from 

250-990, with the third digit always reported as zero. Area Examination 

scores cannot be compared across areas. Scores may appear t<^^fe|^irailar, 
but a particular score does not necessarily represent the same ley©! of 

proficiency from test to test. The test format is multiple* choice qb^stion^* 
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that measure principles and concepts from teacher education. programs (BTS, 
1981). Scaled scores are assumed to approximate interval data. 

The coefficient of reliability (Kuder-Richardson 20) for the WCET 
is reported to be .96**, with a standard 0rj^>r v of measurement of 21. The 
Area Examinations have reliabilities ranging from .913 to .953 (Ruder-Richardson 
20), with a staridard error of measurement from 20-29 for tests that were ^ 
used for this research (ETS f 1978). Validity of the tests is expressed 
as a comparison between the content of thV^jte^ta.and what is included in 
teacher education programs. The tests were pregjlred after extensive discussions 
with teachers and teacher eduqators, and examination of • course materials 
(ETS, 1981). 

Georgia Teacher Certification Tests . The Georgia tests are criterion- 
referenced tests that assess an individual's knowledge of content in his 
or her' teaching field. The tests were developed as part of the perfor«ance- 
based certification policy of the State of Georgia. Cut-off -scores and 
minimal performance standards were set in 1977-1978 by the Department of 
Education, with cut-off scores two an* one-half standard errors of measurement 
below the determined minimal level. The tests were designed to reflect . 
the instructional content of Georgia public schools. A large number of 
" teachers were included in the process to review objectives of the tests 

in order to maximize the degree of content validity. The reliability data 

k 

are expressed as Kuder-Richardson 20 coefficients. The coefficients on 
the tests range from 0.85 to 0.94. The final scaled score is an adjustment 
of raw scores so that the same cutting score of 70 on all of the tests 
corresponds to different percentage correct values, depending on the test 
taken (Georgia Teacher Testing Program. .., 1981 ). 



Teacher Performance Assessment Instrument . The TPAI is used for • 

an on-the-job assessment of teaching plans and materials, classroom £K>cedurea 

and Interpersonal skills, as demonstrated by a teacher in an actual teaching 

situation. v 

The development of the TPAI began with a search of the literature 

% 

related to teaching competencies and their validation. A large dumber 

- I 

of Georgia educators, reacted to an initial list of competencies Aand those 
rated as "essential" by a majority were used. 

The Instrument is composed of 14 generic competencies (see Appendix A). 

\ 4 

These competencies are assessed by 45 indicators which have four! or 
five descriptors each. A team of three data collectors assesses 1 eaoh teacher 
twice during the school year. The team is composed of an administrator 
in the school (such as the principal), a peer, and a data collector from 
outside the school system. All are trained to use the Instrument in a 
50-hour program. The program includes study of the instrument in detail, 
along with experience at rating portfolios, interpreting written interview 
data, interviewing peers, and rating classroom videotapes. Trainees Undergo 
a proficiency check at the end of the training program (Caple et al., 1979; 
Caple, 1981). A teacher must demonstrate mastery on all 14 of the competencies. 
Mastery is attained when 75 percent of the indicator ratings are at or 
above a prescribed minimum level over two consecutive assessments. Three * 
years are allowed to demonstrate mastery (Georgia Department of Edjjoatio£, * 
1981). Interrater agreement rates of 60 peroent are reported after viewing 
videotaped lessons, but agreement rates in excess of 80 peroent have been 
notec} for In-the-olassroom observations (Caple, 1981). The content validity 
of the Instrument is based on the development of the instrument from compe- 
tencies that had been affirmed as "essential" for effective teaohing. 
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Construct* validity studies have resulted in three strong factor solutions: 
• * 

planning, learning environment, and dasspoerfD management. Combined, these 

factors accounted for 60 percent of the common variance in the set of TPAI 

scores. Criterion-related validity studies .have been carried ofrt with 

criterion measures, Such as learning environment, pug^tengagement, and 

achievement gains. Two studies rejfWfcMnoderate correlations (up to .7) 

of TPAI measures and learning environment as reported by pupil perceptions. 

A number of studies have sought £o*explore the criterion related validity 

V 

of the TPAI, using pupil achievement as the criterion. Correlations using 
teacher-made tests have produced significant results; with standardized 
measures as the criterion, results have been mixed (Capie, 1983). 

* 

Analyses * * - <— 

Analysis of the relationship of the NTE area scores and the TCT scores 
in corresponding areas. To determine the relationship between the corresponding 
area scores* on the NTE and the TCT, a bivariate correlation analysis was 
performed for eSach area using P*0C CORK (SAS, 1979). P ROC PLOT (SAS, 1979) 
was used to produce scattergrams of the data* for each a^ea. 

Analysis of the relationship of the TCT^raw scores and the NTE sealed 
scor > ,pg~b^ the performance indicators oh TPAI. To determine 1 the relationship 
between the score on the TCT, the NTE area scores, the NTE weighted common 
score, the NTE professional education scone and each performance competency, 
a bivariate correlation analysis was performed for each using Pfi0c4x)RR 
(SAS, 1979). 
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RESULTS 

Georgia Teacher Certification Tests 
and the National Teacher Examinations 

To answer the' question — "What is the relationship between the Georgia 

Teacher Certification Test raw scorp and the scaled score on the correspond- 

ing National Teacher Examination Area Test for a Selected Population?" — 

Pearson-product moment correlations were calculated for each area (Tablfe 

1). (See Appendix B for the complete set of correlations.) The correlation 

TABLE 1 



PEARSON -PRODUCT CORRELATIONS FOR THE TCT AND NTS' 
FOR SELECTED POPULATIONS i 



*rrcT AREA 



NTE AREA 



NTE WEIGHTED 
COMMOK 
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r 


r2 


r 


r2 


Art 


107 


.75 


.56 


.68 


.46 


Business 


121 


- .70 


.49 


x " .73 


♦54 


Communicative Arts 


98 


.68 


.46 


.78 


.61 


Early Childhoodu 


828 


.63 


.40 


> 

.73 




* 










Home Economics 


91 


.83 


.68 


.80 


.64 


Health and Physical Education 


327 


.68 


.47 


.69 


.48 


Industrial Arts 


32 


.82 


.67 


.65 


.42 


Mathematics 


61 


.83 


' .69 


.67 


.45 


Mental Retardation 


274 


.71 ' 


.50 


.80 


• V 


Music 


128 


.82 


.67 


.62 


.39 


Social Studie^ 


164 


.79 


.62 


.77 


.60 


< 


-** 












coefficients for the TCT with th* NTE Area Examinations range from .63 
for v early childhood to .83 for mathematics* and home economics. The range 
for the TCT with th6 NTE Weighted Common Examinations is from .62 for musir 
to .80 for home economics and mental retardation. Commonalities range t 
from 40 to 69 percent for the area tests and from 39 to 64 percent for 
the TCT with fche NTE Weighted Common Examination (WCET) scores. 

A second question — "What are the corresponding scores on the NTE for- 

* 

each population subgroup which passes each Teaoher Certification £est?" — 
led to a comparison of the number of individuals who would pass or "not t 
pass the NTE, if a cut-off score were set on the NTE to' pass approximately 
the same number of persons that passed the TGT. Percentage agreements 
were calculated tor each area, using the ""set dut-of f score on the NTE and 
the existing cut-off score of 70 on the TCT. Percentage agreements ranged 
from 73 percentage agr^fement for 'home eponomics to 97 percentage agreement 
music, v Passage rates for first tedtjpg on the TCT range from 55 to 97 
percent (Table 2). 
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TABLE 2 



NUMBERS OF PERSONS PASSING OR FAILING TCT 
WITH NTE COMPARISONS FOR SELECTED 
GEORGIA POPULATION 



TCT ART-— NTE ART EDUCATION 



N = \Ol 

NTE Cut-off = 530, 27 Percentile* 

* Agreement = 85* 

* Passing TCT = 79* 



Non-agreement 
Fail TCT 
Pass NTE 9 

Agreement 
Fail TCT 
Fail NTE 



TCT BUSINESS NTE BUSINESS EDUCATION 



N = 121 

NTE Cut-off = 590, 4»» Percentile 
% Agreement = 83* 
% Passing TCT = 61* 



Non-agreement 
Fail TCT 
Pass* NTE 11 

Agreement 
Fail TCT 
Fail NTE 



36 



TCT COMMUNICATIVE ARTS NTE ENGLISH LANGUAGE 



N = 98 

NTE Cut-off = 520, 23 Percentile 
% Agreement = 89* 
% Passing TCT ={^% 



Non-agreement 
Fail TCT 
Pass NTE 6 

Agreement 
Fail TCT 
Fail NTE 13 



Agreement 



77 



Pass TCT 
Pass NTE 



Non -agreement 
Pass TCT 
7 Fail NTE 



Agreement 

Pass TCT 
61 Pass NTE 



Non-agreement 
Pass TCT 
10 Fail NTE 



Agreement 

Pass TCT 
74 Pass NTE 



Non-agreement 
Pass TCT 
5 Fail NTE 



TCT EARLY CHILDHOOD NTE EARLY CHILDHOOD EDTJGATION 



Non-agreement 
N = 828 Fail TCT 

NTE Cut-off = 520, 22 Percentile Pass NTE 27 

% Agreement = 93* «• « 

% Parsing TCT = 91* Agreement 

Fail TCT, 
Fail NTE 20 

TCT HOME ECONOMICS NTE *H0ME ECONOMICS EDUCATION 



Agreement 

Pass TCT 
752 Pass NTE 

Non-agreement \ 
Pass TCT | 
29 Fail NTE 



N =' 91 . 

NTE Cut-off = o*20, 50 Percentile 
<% Agreement = 73* 
* Passing TCT =55* 



Non-agreement 
Fail TCT 
Pass NTE 15 

Agreement 
Fail TCT 
Fail NTE N 26 



Agreement 

Pass TCT » 
MO Pass NTE 



Non -agreement 
Pass TCT 
10 Fail NTE 
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TABLE 2 
Continued 

TCT MATHEMATICS NTE MATHEMATICS EDUCATION 



N s 61 * 

NTE Cut-off = 570, 46 Percentile 

* Agreement = 92* 

* Passing TCT = 74* 



Non -agreement 
Fail TCT 
Pass NTE 3 

Agreement 
Fail TCT 
Fail NTR 13 



Agreement 

Pass TCT 
43 Pass NTE 

Non-agreement 
Pass TCT 
2 Fail NTE 



TCT MENTAL RETARDATION NTE EDUCATION OF THE MENTALLY RETARDED 



N = 27 M 

NTE Cut-off = 500, 17 Percentile 
* Agreement = 92* 
*' Passing TCT = 91* 



Non -agreement 
Fail TCT 
Pass NTE 12 

Agreement 
Fail TCT . 
Fail NTE 13 



Agreement 

Pass TCT. 
238 Pass NTE 

Non-agreement 
Pass TCT 
11 Fail JfoTE 



THT MUSIC — - NTE MUSIC EDUCATION 



N = 128 

NTE Cut-off = 480, 11" Percentile 
% Agreement = 97% 
% Passing TCT = 97* 



Non -agreement 
Fail TCT „ 
Pass NTR 2 

Agreement 
Fail TCT 
Fail NTE 2 



Agreement 

P|ass .TCT 
124 Pass NTE 



Non-agreement 
Pass TCT 
0 Fail NTE 



TCT HEALTH AND PHYSICAL EDUCATION NTE PHYSICAL EDUCATION 



N = 327 

NTE Cut-off = 610, 49 Percentile 
% Agreement = 77* 
* Passing TCT = 60* 



Non -agreement 
Fail TCT 
Pass NTE 39 

Agreement 
Fail TCT 
Fail NTE 93 



Agreement . 

Pass TCT 
159 Past NTE 

Non-agreement - 
Pass TCT 
36 Fail NTE 



TCT SOCIAL STUDIES — NTE SOCIAL STUDIES 



N = 164 

NTE Cuf-off = 520, 25 Percentile 

* Agreement = 87* 

* Passing TCT = 86* 



Non -agreement 
Fail TCT 
Pass NTE 10 

Agreement 

Fail TCT , 
Fail NTE 13 



Agreement 

Pass TCT 
130 Pass NTE 

Non -agreement 
Pass TCT 
11 Fail NTE 
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•ETS. NTE Program Data, 1978-1981. 
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Mean scores on the NTE area tests for Georgia students are higher 
than the mean scores for th« population represented by 1 ,he accumulated 
program data, reported by NTE (Table 3). Scattergrams show the distribution 
for each subject area (Appendix C). Note that scales differ for each subject, 
therefore, care should be used in visual comparison from one graph to another. 

TABLE 3 



SCORES ON SELECTED NTE AREA TESTS FOR 
GEORGIA AND THE NATION 



NTE AREA TESTS 



NATIONAL MEAN* 

u 1 0 



•Based on data for 1978-81 NTE program data 
♦Based on data for 1978-81 Georgia population 

Note: Scores cannot be compared across areas 



GEORGIA MEAN+ 



Art Education 


579 


592 


Business Education 


593 


fi15 


Early Childhood Education 


593 


fi30 


Education pf Mentally Retarded 


586 


606 


English .Language and Literature 


580 


596 


Home Economics. Education 


602 


615 


Mathematics 


577 


617 


Music Education 


592 


"632 


Physical Education 


600 


620 


Social Studies 


576 


591 
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Georgia Teacher Certif loatlon Ttests 

and the Teaoher Performance Assessment Instrument 

The relationship between the Georgia Teaoher Certification Tests and 
the Georgia Teaoher Performance Assessment Instrument competencies was 
explored by oaloulating Pearson-product moment correlation:* for the subject 
area subpopulations. Correlations were calculated using the raw score 
on the area test and the percentage of indicator* whioh had been passed 
at the 'Specified minimum level for eaoh competency. Possible raW soore 
ranges on the TCT were from 0 to approximately 110, depending on the test. 
Ranges on the TPAI were from 0 to 100 peroent for eaoh competency. Correlations 
are shown for both the spring and fall assessments, with a range of -.27 
to. .45 (Appendix D). Of particular interest is oompetenoy number 10— "Demon- 
strates an understanding of the sohool subject being taught and demonstrates 
its relevanoe." The correlation ranges from -.10 to • 3*5 for the fall assessment, 
and from r»07 to .3^ for the spring assessment. These data must be examined 

with the notion that the oompetenoy is composed of two indicators: "Helps 

\ 

learners recognize the purpose and importance of topios or activities," 

and "Demonstrates knowledge in the subjeot area." Therefore, when interpreting 

the correlation of the content tests with the performance" asessment oompetenoy , 

not only, knowledge' of the subjeot matter is assessed. In addition, the 

minimum standard for the first indicator is that the teaoher oonveys "th* 

purpose or importance of most topios or activities studied." 

National Teaoher Examinations and the 
Teaoher Perfor mance Assessment Instrument 

The competencies r>n the TPAI were correlated with soaled soores on 

the NTE Area Examinations, the professional eduoation examinations whioh 

are a part of the Common Examinations, and the Weighted Common Examinations 

(WCET). Ao with tho TCT correlations, the percentage of indicators that 




were at the minimum level or above was used as one variable, with the scaled 
scores on the respective test used as the other variable. Correlations 
are shown for both the fall and spring assessments for the respective areas, 
with a range of -.12 to .52 (Appendix E). Of particular interest are competency 
number 10 correlations with the NTE area tests, with ranges from .23 for 
physical education to .33 for mental retardation in the fall, and .02 for 
mental retardation to .25 for early childhood in the spring. 
Teacher Performance Assessment Instrument Mastery versus Non-mastery 

Mean TCT and NTE scores for those teachers who did or did not demon- 
strate mastery of all Ij^TPAJ competencies during their first year of teaching 
are shown in Tables 4 and 5. In addition, percentages of those teachers 
demonstrating or not demonstrating mastery are shown for each subject area. 

TABLE M ' 



MEAN NTE SCORES FOR MASTERY VERSUS NON-MASTERY GROUPS ON THE TPAI 
FOR FIRST YEAR TEACHERS WHO GRADUATED FROM UNIVERSITY SYSTEM OF GEORGIA 

\ 

MASTERY NON -MASTERY 

EARLY CHILDHOOD EDUCATION N = 139 (78*) N = MO (22%) 

NTE Area Teat Mean 636 610 

NTE Professional Education Mean 59.3 56. M 

NTE WCET Mean 567 5*»3 

EDUCATION OF THE MENTALLY RETARDED N = 43 (70*) ^' * N a 18 (30*) 

NTE Area Test Mean 638 582 

NTE Professional Education Mean 60.9 5M.6 

NTE WCET Mean 581 532 

PHYSICAL EDUCATION N = 19 (29*) , N : 16 (71*) 

NTE Area Test Mean 611 625 

NTE Professional Education Mean 50.6 M9.3 

NTE WCET Mean - 532 516 

Note: Area test scores cannot be compared across subject areas. 
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TABI.K 5 * • 

MEAN TCT SCORES FOR MASTERY VERSUS NON-MASTERY OROUPS 

ON THE TPAI FOR FIRST YEAR TEACHERS 

MASTERY NON-MASTERY 

MI N B 18 (51*) N - 17 (49*) 

_ TCT Raw Soore Mean 93.0 89.0 

~ TCT Soaled Score Mean Jf\.?. 73.9 ~ 

COMMUNICATIVE ARTS N * 17 (56*) N * 37 (44*) 

TCT Raw Soore Mean flU.9 82,5 

TCT Soaled Soore Mean 1*71.3 72.3 

EARLY CHILDHOOi) N n 319 (71$) N a 133 (29$) 

TCT Raw Soore Mean 86.9 80,6 

TCT Soaled Soore Mean 79.1 73.8 

HOME ECONOMICS N = 18 (51$), ; N ■ 17 (19$) 

TCT Raw Soore Mean 85.3 81.6 

TCT Soaled Soore Mean 78.3 77.7 

HEALTH AND PHYSICAL EDUCATION N = 99 (72%) N ■ 39 (28$) 

TCT Raw Soore Mean 92.3 88.2 

TCT Soaled Soore Mean 72.7 69.4 

MATHEMATICS N « 20 (48$) N - 22 (52$) 

TCT Raw Soore Mean 78.1 76.2 

TCT Soaled Soore Moan 77.6 76. 0 

, MENTAL RETARDATION N a 86 (70$) N m 37 (30$) 

TCT Raw Soore Mean 86.0 8 1.9 

TCT Soaled Soore Mean 77.1 73.5 

MUSIC N = 22 (35$) N a MO (65$) 

TCT Raw Soore Mean 91.7 92.6 

TCT Soaled Soore Mean " 82.5 83.3 

SCIENCE N « 31 (16$) N ■ 37 (54$) 

TCT Raw Soore Mean 77.8 

TCT Soaled Soore Mean 77.3 75.1 

SOCIAL STUDIES N a 35 (16$) N a 11 (51$) 

TCT Raw Soore Mean 88.0 81.5 

TCT Soaled Soore Mean 80.6 21 77.7 

O 

ERXC Note: Raw aoorM cannot *>* compared aoroaa aybjeot areas. 
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SUMMARY AND CONCLUSIONS 

National Teacher Examinations and 
Georgia Teacher Certification Testd 

The correlations between the National Teacher Examinations Area Tests 
and the corresponding Teacher Certification Tests, in the range of .63 
to .83, indicate that the common variance of the tests is from MO to 69 
percent. The tests do not purport- to measure the same information — the 
NTE tests the content of teacher education programs, and the TCT measures 
content of the curriculum in Georgia public schools — yet both tests do 
cover substantive knowledge of fields taught in the schools. An examination 
of the bbjectives of both tests on early childhood, which have the lowest 
correlation of those examined, reveal that the NTE is testing growth, develop- 
ment, and learning, along with curriculum organization and activities for 
young children. The TCT tests for the development of children" and their 
activities, and for basic knowledge in language arts, mathematics, science, 
social studies, art, music, health, and physical education. It is interesting 
to note that, for this particular comparison, the correlation of the TCT 
with the NTE Common Examinations is higher than with the NTE Area test, 
a possible explanation being the emphasis on basic subject areas. The 
TCT includes a number of general education objectives. The two mathematics 
tests have the highest correlation. From the objectives of both tests, 
the content of the TCT is fairly consistent with the NTE. However, the 
NTE includes questions on professional education and the history of mathematics 
to a greater degree than the Georgia test. A closer examination of the 
content of the NTE tests with the state tests might be warranted. 

The examination of corresponding scores on the NTE for those persons 
passing or not passing the TCT relate* to a number of issues. The percentage 
of agreement between the two tests on numbers of persons passed or failed 



(when an NTE cut-off score is used that passes the same number of persons 

as the TCT) ranges from 73 to 97 percentage agreement. The tests are 

essentially being used to screen out those who cannot perform at a particular 

standard. The results of this study help decide whether or not another test 

could lie used for reciprocity agreements if percentage agreement of numbers 

of persons passing one state test with the other test is relatively high. 

It is interesting to note that all of the areas which were examined fall 

into a similar narrow range of correlation coefficients and percentage agreements 

on pass/fail. Bight of the 10 areas (Table 2) fall into the 83 or higher 

category on percentage agreement, with four at or above 92 percentage agreement. 

Does each state have a unique knowledge base on whtafr teachers must 
be tested? The correlations for the Georgia population indicate relatively 
high conmon variances between the NT$ and TCT. Are states willing to relinquish 
the supposed uniqueness of a state-developed test so that certification could 
be facilitated for out-of-state teachers who have taken a different test? 

Iiv using standards, should the states not look at the standard of 
quality that has been set by their testing requirements? Are levels set 
equitably for different subject areas? "TOft standards of states using testing 
are now set at minimum levels, with only those persons at t^he bottom being 
screened out. Is it important for states that are uaingVsome type of testing 
program to create a data base for monitoring their population in terms 
of the exclusion for any one subject area to a greater degree th^n any 
other? Shouldn't the states have some type of yardstick to permit suoh 
HK>nitoring? In this study, when NTE cut-off scores N were arbitrarily set 
so that similar numbers of persons would "pass" the tKo tests, national 
percentiles varied from the 11th to the 50th percentile. Six* of the test 
areas were at or below the 27th percentile (ETS, 1981)., Why does this 
great variation exist? Differences in passage rates for prospective teachers 

20 



of different subjects are puzzling. For the different areas v the TCT pass 
rates on first test takers range from 55 percent on home economics to 9^ 
percent on the music test (Table 2). Overall passage rates upon retakes 
will be higher. 

SREB states should definitely examine the data from their test populations 
to decide whether or not reciprocity agreements could be worked out tor 
those teachers who migrate into or within the region. Examination of the 
Georgia population data supports the riqtion that reciprocity could be worke^ 
out among the SREB -states, using tests that are currently being used or 
are currently available. 

Because of the 6x post facto design of the study ; limitations are 
inherent. The reported high reliabilities of the TCT and NTS instruments 
mean attenuation problems were at a minimum (Thorndike, 1971). Errors 
in measurement tend to make correlation coefficients smaller than they 
would be if no measurement errors existed. In addition, extraneous variables 
that wer£ not accounted for, such as time jlifferences in taking the two 
tests or the fact that the TCT is required for certification purposes 
and the NTE is not, could have Influenced test scores. Results should 
be examined with those factors in mind. 

National Teacher Examinations and 
the Teacher Performance Instrument . 

Pearson -product moment correlations between the NTE weighted Common 

test, the professional, eduoatlon subtest, and the Area test for th l e three 

subpopulatlons are mixed, but generally low. Of particular interest is 

competency 10 because it concerns the relationship between a paper-and- 

pencil test on subject matter being taught JLn the schools, and an assessment 

of whether or not the teacher shows competency in demonstrating *n understanding 

of the subjeot matter and conveying the importance of most activities or 
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topics studied to the learners. The fall correlations for the Area tests 
range from .23 on physical education to .33 for mental retardation. These 
correlations tend to support previous studies indicating that tht common 
variances between NTE tests and measures of teacher performance are low* 
^ The findings would support ,the view of ETS and the Georgia Department of 
Education that knowledge is only one part of the complex process called 
teaching. At the same time, ETS questions the present measurement of "teaching 
success" because of the instrumentation, and the observation that "teacher 
behaviors themselves tend to be somewhat unreliable" (ETS f 1978, p.1). 

Georgia Teacher Certification Tests and 

the Teacher Performance Assessment Instrument 

Examination of the Pearson-product correlations between the Georgia Teacher 

\ 

Certification Test and the TPAI for each subject area are ^qUite mixed. * 
They range from -.20 to .45 for the fall assessment. For competency 10 f 
the academic subjects tehd to show positive correlations for the fall, * 
although very weak, with r^rtges froar .04 in communicative arts to .35 for 
mathematics, indicating a common variance from approximately one to 12 
percent. The sub^Qts of art, home economics, and music all show negative 
correlations, very weak, with ranges from -.03 for music tJ -.10 for home 
economics, indicating common variances of under one percent. One limitation 

a, 

of the comparison of the TCT with the TPAI is that those persons who did 
not pass the TCT will most likely not be teaching, although a one-year 
grace period is allowed before passage, of the TCT. 

No definitive answer has come from this study on whether or not a 
paper-and-pencil test and the ability to put the knowledge across are related. 

The data suggest £wo possibilities: either content knowledge, as measured 

i 

by the TCT or NTE, accounts for only a small percentage of teacher performance; 
or the performance instrument is not differentiating between those who 
_ ®_ do or do not perform. 
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Research concerning the use of these performance assessment instruments, 
especially for certification decisions, is very limited. We note the differ- 
ential percentagejKof teacherjguof different subject areas reaching mastery 
of the TPAI after the two assessment of the first year (Table <Land Table 
5) % 'i)o the differences in passage rates for different subjects continue 
to exist after the three years that are allowed for mastery? If so, why 
are differences present? In the summary of the mastery versus non-mastery 
groups (With the exception of music for TCT, and the. physical education 
NTE area test) teachers who complete mastery in one year have slightly 
higher mean scores than those who do not. Does this continue to be the 
6ase after the three -year period? 

Performance ratings in general have been attacked, and studies involving 
^he correlation of those ratings with student performance as measured by 
teacher-made and standtrdized tests have been mixed (Coker et al«t 1980; 
Capie, 1981). Differences in the ability of teacher observation instruments 
to measure teacher performance have been suggested (Coker, et al«, 1980; 
Dickson and Wiersma, 1980). Additional research needs to be conducted 
concerning the ability of these instruments to differentiate teacher performances 
which are linked to student learning and attitudes. 

Conclusions ' f 

The public is demanding accountability. The perception that large 
numbers of unqualified students are graduating from oollege and moving 
into teaching positions has led states to mandate tests for teachers and, 
| in some cases, some kind of on-the-job assessments. Across the region 

approximately 5 to 15 percent of prospective teachers are being eliminated. 
The question of the ultimate impact these programs will have on students 
remains to be seen. The existing research base is not consistent as to 
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the relationship between teacher testing arid student criterion, or teacher 
performance assessment and student outcomes. 

States are spending large amounts of money to ornate and validate 
tests of knowledge for teachers, with the idea that a minimum knowledge 
level essential. It appears from the data in this study 0 that reciprocity 
could be worked out for teacher testing in the'SREB states, using existing 
state and/or national tests. States should monitor results of testing 
carefully, examining passing percentages for subject areas and for minority 
representation. \h * 

The' question of whether or not performance assessment is accomplishing 
its "goal needs further study. Previous research and this study tend to 
support the idea( that "a teacher's knowledge base, as measured by a paper- 
and -perifcil test, does not haye a linear relationship with the ability to 
"put it across. 11 However, the instrumentation used to assess teachers 
in this and other studies must be examined to a greater extent. Just as 
with testing for knowledge, it appears that the chances of mastering the 
performance competencies depend somewhat on the subject matter field. 

Many states now apply a combination of paper -and -pencil tests and 
performance evaluations to separate those teachers Who are believed to 
be competent from, those who are not. Two vital questions need further 
research: Does the demonstration of a combination^^ paper-and-pencil 
knowledge and performance skills enhance the overall quality of teachers? 
What performance skills does an effectives teacher possess? the bottom 
line should be student learning and attitudes. States should contiriually 
attempt to gather data on the processes they have in place and refine those* 
methods, if necessary. * 
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APPENDIX A 

Teacher Performance Assessment Instrument: Competencies 

* * « 

Teaching Plans and Materials 

TPM I: Plans instruction to Achieve selected objectives 

TPM II: Organizes instruction to take Into account individual differences 
among learners # 

TPM III: Obtains aqd uses information about the needs and progress of 
-i individual learners ' . 

TPM IV: Refers learners with special problems to specialists 

TPM V: Obtains and uses information about the effectiveness of 
% f instruction to revise it when necessary 

Classroom Procedures Instrument ■ 

CP VI: Uses instructional techniques, methods, and media related to 

the objectives , 

CP VII: Communicates with learners 

CP VIII: Demonstrates a repertoire of teaching methods 

CP IX: Reinforces and encourageg learner involvement in instruction 

CP X: Demonstrates an understanding of the school subject being taught 
and demonstrates its relevance € 

<* 

CP XI: Organizes time, space, materials, and equipment for instruction 

v * 

— Interpersonal Skills Instrument 



IS XII: Demonstrates enthusiasm for teaching and learning and the subject 
being taught 

IS XIII Helps learners develop positive self-concepts 4 
IS XIV: Manages classroom interactions 
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APPENDIX B 

Correlations of Georgia Teacher Certification Tests (TCT) 
with the National Teacher Examinations (NTE) 
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APPENDIX C 

* Scatter grams of Correlations Between* NTE and TCT Test Results In Selected Subjects 
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TCT 



X 
2 

3. 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 
14 



Science 
N » 68 



Fall 
. 31773 

.17518 

:05264 

.00156 

.14430 

.21679 

. 33549 

. 09354 

.17282 

. 159J7 

.12551 

.12977 

.25290 

.22388 



Spring 
.1007^ 

.11296 

.016Q6 

.09604 

-.11165 

.02314 

„03165 

-.04301 

-.04112 

01778 

-.17272 

-.05278 

-.09469 

-.26521 



*** 
N * 35 



Communicative 
Arts^ 

N * 84 



Fall 

.25881 

-.00914 



.01703 
.00254 
.12196 
.14357 
.05786 



Spring 
.32522 



Fall 
.09409 



.075^5 (-.13975 
.09560 K21539 13243 
.06504 
.08954 
.04615 
.01431 



.1699^ - 
.04308 
.00979 - 
.10274 - 
.24169 



.08118 j .09534 )-, 
.08307 -.07065 . 
.07791 L. 25258 K 
.01573 ,13344. 
.07927 -.03417 



i 



.00920 I .04895 



I- 



.02468 
13190 
03862 
13650 
00121 
10770 
15666 



Spring 
.28958 

.1367* 

.15793 

.22853 

.21423 

.07961 

.11330 

.00940 

.03241 

.25783 

.11*277 

.05885 

-.02327 

.12152 



Early* 
Childhood 



Fall 

.25830 

.28961 

.12052" 

.12121 

.21661 

.24669 

.32475 

.12313 

.18564 

.24610 

.03508 

. 22277 

.22456 

■07279 



Spring 
.28215 

.25208 

~. 17115 

.11944 

.26945 

.26867 

.41924 

.14846 

.26078 

.33572 

.09265 

.31786 

.23777 

■13463 



Borne 

Economl cs. 

K - 35 



Fall 
.13101 

,13837 

.02585 

.08334 

.16332 

.18741 

.17887 

.35306 

.04.462 

.09699 

.05700 

.14051 

.25447 

.20417 



Spring 
-.01328 

.31157 

.26649 

a 1890 

'.06264 

.15529 

.25120 

.21180 

.01886 

.01799 

.14708 

-.24433 
.09495 

-.04547 



Mathematics 

N « 42 . 



Fall 
.08041 

. 20957 

.13487 

.25755 

. 15966 

.14592 

.45325 

.13031 

.24821 

.35323 

.19804 

.13637 

.38681 

.05642 1 



Spring 
.05687 

.23372 

.04079 

.11883 

..13701 
.03862 
.20520 
.05188, 
.08540 
.07964 

-.14162 

-.03512 
.27346 

-.05889 



Music 
N - 62 



Fall 
.00606 

.08640 

h. 01660 

04529 

07209 

K 06675 

.06235 

16791 

-.11728 

02590 

K 01689 

.03134 

.04974 

K 13729 



Spring 
.1*357 

-.05108 

.01277 
-.09327 
-.02665 
-.04694 

.01093 
-.11162 
-.03275" 

.17585 
-.05988 
-.04712 

.00179 
13515 



Mental 
Retardation 

* S 123 



Fall 
.25131 

.35233 

.03428 

.26*31 

.29277 

.36723 

.21899 

.24733 

. 29525 

.32002 

.16183 

.23060 

.21497 

.14712 



Spring 
.28731 

.25241 

.16758 

.22943 

.21436 

.15826 

.30498 

.20063 

.22224 

.16946 

.17465 

. 29098 

.15230 

.08957 



Physical" 
Education ^ 

N - 138/ 



Fall 
.28403 

.26164 

.24729 

.10411 

.32453 

.17026 

.33479 

. 16666 

.19168 

.34748 

.08999 

. 19790 

.17544 

,03346 



Soring 
V30865 

.26333 

'.34279 

.25978 

.29192 

.19065 

.29784 

.11337 

.12926 

.25977 

.24508 

.26148 

.22103 

.16792 



Social 
Studies 

N » 76 



Fall 

.02571 

.00909 
.09481 
.02996 
.03736 
.04293 
.19183 
.09074 
. 25686 
.25218 
,"12628 
.16948 
.04823 
.17248 



Spring 
.21944 

.14524 

.08590 

.14858 

.14527 

.24299 * 

.248^5 

.01885? 

.21476- 
I 

.31083 
.1415l| 
.07857 
.10847 
.01742 
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u 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 



NTE AREA 



Early Childhood 
N - 179 



Fall 

.22765 

, 31190 
.17485 
. 10606 
. 20908 
.25402 
.29209 
•12353 
.21139 
.27797 
.08877 
.23988 
.19789 
.17100 



NTE PROFESSIONAL 
EDUCATION 



NTE WCET 



Spring 
.21855 

.04281 

.01141 

.06905 

.02908 

.31282 

. 35266 

. 10503 

.22801 

.24794 

.05650 

.25170 

.22032 

.21612 



Fall 

.16984 

.23150 
.15952 
.04953 
.18596 
.17761 
.24280 
.08783 
.15486 
. 24161 
.04336 
.23062 
.14323 
.12~400 



Spring 
.23325 

.06297 

.03679 

.06234 

K 01223 

.28647 

. 28390 

.09141 

20848 

24733 

K 00792 

.18879 

.14539 

.15519 



Fall 

.14594 

. 21658 
.11826 
.06398 
.18462 
.18215 
. 25041 
.07372 
.14151 
. 22010 
.04449 
.21043 
.13716 
.14603 



Spring 
.21176 

r. 09530 

.02704 

.05746 

.016QQ 

.24568 

.28568 

.09221 

r 

. 19662 
.21702 
.03334 
.23418 
.16898 
.13886 



Mental Retardation 
N - 61 



NTE AREA 



Pall 

.30089 

.50496 
.26282 
.40012 
.47546 
, 37694 
.19063 
40238 

.32654 
.20629 
.26373 
.06459 
.24688 




Spring 
.20483 

.24253 

.08563 

.19670 

.03332 

-.00573 
. 31654 

-^02*92 
.13070 
.02215 
.11468 
.18386 
.0ol33 
. 14058 



NTE PROFESSIONAL 
EDUCATION 



NTE WCET 



Fall 

.23068 

.44420 
.31564 
.43376 
.51648 
.28785 
.18881 
.34487 
.28500 
.36X16 
.14828 
.31048 
.09584 
. 21894 



Spring 
.14117 

.17156 

.07047 

.14980 

.01065 

'.00608 

. 33126 

.09729 

.03289 

•04451 

.02410 

.19062 

.09857 

.02939 



Fall 

.27465 

.36825 
. 20568 
.44069 
.47186 
.33344 
. 20729 
.24557 
.29809 
.31776 
.12493 
.25823 
12427 
31922 



Spring 

. 20097 

.23492 
.18450 
.26990 
.03434 
.04144 
.43834 
.11596 
.03649 
.12513 
.01111 
.24379 
.16003 
.13641 



Physical Education 
N - 65 



NTE AREA NTE PROFESSIONAL NTE WCET 



Fall 

.08456 

.06680 
.10236 
.09611 
.27077 
.14421 
.28076 
.19208 
.15469 
.23126 
.08650 
.06718 
.08498 
032 



Spring 
-.02654 

-.02570 

.07712 

-.04708 

-.05018 

-.05559 

.01979 

-.01953 

.00325 

.13281 

.05002 

-.01779 

.04736 

.05485 



Fall 

.22797 

.10542 
.17391 
.05443 
.24866 
.25730 
.32437 
.23737 
. 23099 
.30246 
.20119 
.16371 
09869 
09133 



Spring 
.09116 

.14433 

.239 33 

.07118 

.16566 

.10565 

.12973 

.09847 

.08972 

. 24350 

.15680 

.13998 

.05660 

.18684 



Fall 

.19648 

.12159 
. 21186 
.01472 
.30654 
.28754 
.36030 
.25677 
.25588 
.34614 
.20481 
14311 
.08710 
17258 



Spring 

.13255 

.16208 
.26950 
.05454 
.1422$ 
.12872 
.15959 
.15655 
.07978 
.28070 
.11153 
.11566 
.15301 
.18623 
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